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ABSTBACT 


Military  aircraft  missions  are  all  multidimensional  in 
nature.  This  means  that  every  mission  can  be  divide!  into 
usually  one  overall  goal  or  purpose  (i.e.,  destroy  the 
target,  deliver  the  supplies,  rescue  the  survivors,  etc.), 
with  several  subgoals  (safety,  minimize  susceptibility, 
timeliness,  etc.).  Since  missions  are  multidimensional,  the 
operator  effort  in  the  form  of  mental  and  physical  action 
(performance)  becomes  multidimensional.  The  multidimentional 
nature  of  skilled  aircrew  performance,  in  turn,  requires 
that  several  criteria,  all  of  which  are  relevant  for  a 
particular  activity,  be  defined  and  used  [Hef.  1]. 

The  unique  situation  of  an  aircrew  flying  an  aircraft 
for  a  specific  mission  and  the  necessary  determination  of 
subcriteria  for  evaluating  accomplishment  of  that  mission 
requires  further  research  of  an  analytical  and  empirical 
nature.  The  relationship  among  altitute,  airspreed, 
operator  activity,  and  the  hundreds  of  other  system  vari¬ 
ables  that  comprise  the  total  system  must  be  compared  to 
mission  success  in  quantifiable  terms. 

This  study  is  an  effort  to  improve  acquisition  of 
training  performance  information  in  affortable  ways  on 
behalf  of  the  Hellenic  Air  Force  (HAF).  Thus,  it  is  divided 
in:  (1)  The  principles  of  human  performance,  (2)  Definition 
of  the  criteria  and  their  measurement,  (3)  Systematic  defi¬ 
nition  of  performance  measure  appropriate  to  combat- training 
needs,  (4)  Definition  of  a  cost  effective  measurement  system 
usable  in  combat-crew  training  environments  to  acquire  and 
process  needed  training  information. 

The  method  that  was  used  was  a  search  of  the  available 
materials  found  in  the  NPS  library.  Most  of  the  data  found 
was  based  on  OSAF  studies. 
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I.  INTRODUCTION 


The  Hellenic  Air  Force  { H A F)  is  interested  in  optimizing 
its  combat  readiness,  and  consequently  the  combat  readiness 
of  its  combat  pilots.  Greece  has  common  borders  with 
hostile  countries  and  is  constantly  threatened  with  possible 
invasion.  There  are  continuous  violations  of  the  aerial  and 
sea  space  of  Greece  by  neighboring  countries'  airplanes  and 
ships  respectively.  We  (the  Greeks)  must  offset  our 
numerical  inferiority  with  technology  expressed  in  higher 
skilled  combat  pilots.  It  is  imperative  to  increase  to  the 
utmost  the  combat  readiness  and  effectiveness  of  the  HAF. 
In  this  way  we  shall  be  a  strong  power  and  our  enemies  will 
never  try  to  invade  our  country.  Thus,  we  shall  remain  in 
our  purpose,  that  is,  united  and  uninvaded  country  and  most 
important,  because  of  our  power  we'll  keep  the  peace. 

Today  the  need  for  combat  effectiveness  is  more  pressing 
than  ever  before.  We  live  in  a  period  of  rapid  and 
continuous  change  in  technology  and  conseguently  in  our 
tactics.  The  new  generation  of  combat  aircraft  reguire  more 
training  because  of  the  complexity  of  the  new  systems. 
Eecause  of  these  technological  changes  it  is  required  that 
new  changes  in  air  tactics  must  be  established.  Thus, 
because  of  the  above  continuous  changes,  it  is  required  that 
new  criteria  for  combat  readiness  of  combat  pilots  must  be 
established  (created).  Of  course,  HAF  has  established 
criteria  for  its  combat  pilots  as  a  result  of  the  entry  of 
the  present  generation  of  combat  aircraft.  However,  it  may 
be  necessary  to  see  if  there  are  any  critical  points  that 
need  to  be  corrected  or  changed.  For  these  reasons  this 
study  has  been  conducted  in  which  the  US  Air  Force  was  used 
as  a  model  and  a  later  study  will  be  necessary  to  adapt 
these  models  to  the  Greek  reality. 


Thus,  this  study  will  define  combat  readiness,  the 
criteria  and  their  considerations  and  human  performance,  in 
relation  to  the  combat  readiness  of  combat  pilots. 

A.  CCM3AT  READINESS 

But  what  is  combat  readiness?  In  order  to  answer  this 
question,  we  first  must  define  the  word  readiness,  secondly 
combat  readiness,  and  third  readiness  categories. 

Readiness  means  the  quality  of  being  ready  to  act, 
respond,  comply,  etc;  Also  readiness  is  a  state  or  fact  of 
being  ready,  or  prepared;  as  to  have  everything  in  readiness 
for  a  sudden  departure  [Ref.  2].  A  crew  or  crew  member  is 
combat  ready  when  it  is  certified  as  ready  for  combat 
[Ref.  3]. 

1  *  Readiness  Categories : 

Combat  readiness  is  expressed  with  the  following 
standard  categories  and  meanings  [Ref.  4]. 

a.  Fully  Combat  Ready. 

A  unit  fully  capable  of  performing  the 

mission  (s)  for  which  it  is  organized  or  designed. 

b.  Substantially  Combat  Ready. 

A  unit  capable  of  performing  the  mission  (s)  for 
which  it  is  organized  or  designed,  but  having  minor  defi¬ 
ciencies  which  could  reduce  its  effectiveness  or  its  ability 
to  conduct  sustained  operations. 

c.  Marginally  Combat  Ready. 

A  unit  with  major  deficiencies  of  such  magnitude 
as  to  severely  limit  its  capability  to  perform  the 
mission  (s)  for  which  it  is  organized  or  designed,  but 


capable  of  conducting  limited  operations  for  a  limited 
period. 

d.  Not  Combat  Ready. 

A  unit  not  capable  of  perfoming  the  mission  (s) 
for  which  it  is  organized  or  designed. 

Definitions  of  readiness  and  combat  readiness 
were  given  above.  A  classification  of  combat  readiness  into 
four  categories  was  made  in  an  effort  to  specialize  and 
highlight  it.  In  order  to  define  and  understand  combat 

readiness  better  we  have  to  define  criteria  and  their 
considerations,  that  is  to  say,  definition  and  purpose  of 
criteria,  the  categories  of  criteria  and  characteristics  of 
good  criteria. 

E.  CRITERIA  AND  CONSIDERATIONS 

1 .  Definition  and  Purpose  of  Criteria 

Criteria  are  standards,  *  rules,  or  tests  by  which 
measures  of  system  behavior  are  evaluated  in  terms  of 
success  or  failure,  or  to  some  degree  of  failure.  The 
purpose  of  human  performance  criteria  is  to  provide 
standards  or  baselines  for  evaluating  the  success  or 
failure,  goodness  or  badness,  or  usefulness  of  human 
performance  [Ref.  5]  and  [Ref.  6]. 

Criteria  should  not  only  define  the  unique  manner  in 
which  the  operator  should  perform  a  task,  but  should  define 
the  performance  objectives  of  the  entire  man-machine  system 
[Ref.  7]  and  [Ref.  8]. 

2.  Categories  of  Criteria 

The  classification  of  criteria  can  be  accomplished 
from  a  measurement  standpoint;  beginning  with  the  smallest 
known  entity  and  ending  with  the  "ultimate'*  quantity  that 
may  exist.  Several  categories  identified  are  listed  below: 


(1)  Parametric  referent  or  standard  of  performance 
which  is  sought  to  be  met  by  the  operator  or  system. 
Example:  maintain  1000  feet  of  altitude  [Ref.  7]  and 
[Bef.  9]. 

(2)  Parametric  limit  about  the  parametric  standard 
within  which  the  operator  or  system  is  required,  or 
seeks,  to  remain.  Example:  maintain  plus  or  minus 
100  feet  while  at  1000  feet  altitude  [Ref.  7]  and 
[Bef.  9]. 

(3)  System  component  criteria  which  distinguishes 
the  relationship  between  system  components  and  system 
output.  Example:  "  least  effort  "  measured  from 
the  pilot  in  relation  to  maintaining  altitude 
[Bef.  10]. 

(4)  Test  criterion  used  to  evaluate  overall  human 
ability,  usually  expressed  as  a  single  overall 
measure.  Example:  subjective  judgement  of  instructor 
for  a  student  as  to  "pass"  or  ''fail”. 

(5)  Ultimate  criteria  are  multidimensional  in  nature 
and  represent  the  complete  desired  end  result  of  a 
system.  This  category  of  criteria  is  impossible  to 
quantify  due  to  the  multidimensional  nature  of  the 
system's  purpose,  and  hence,  is  a  theoretical  entity 
that  must  be  approximated.  Example:  Any  aircraft's 
mission  [Ref.  1]  and  [Ref.  11]. 

It  may  be  noted  that  all  five  categories  of  criteria 
can  be  either  quantified  or  approximated  in  some  manner, 
with  decreasing  accuracy  as  the  ultimate  criteria  level  is 
reached. 

Obtaining  direct  measures  of  the  ultimate  criteria 
for  a  complex  system  is  seldom  feasible.  This  is  particu¬ 
larly  true  in  military  systems  where  such  criteria  would  be 
expressed  in  terms  of  combat  effectiveness  or  effectiveness 


iii  preventing  a  potential  aggressor  from  starting  a  conflict 

£Bef.  1]. 

Therefore,  it  becomes  apparent  that  we  must  select 
intermmediate  criteria  (types  (1)  through  (4)  above)  in 
evaluating  skilled  operator  behavior. 

3.  Characteristics  of  Good  Criteria 

Using  actual  criteria  as  approximations  of  the  ulti¬ 
mate  criteria  can  be  accomplished  by  several  methods  that 
will  be  presented  in  the  following  paragraphs  . 

Although  there  is  no  certain  method  that  will  lead 
to  the  specification  of  good  criteria,  there  are  seme 
considerations  that  can  be  taken  into  account  which  are 
discussed  below: 

(1)  A  good  criterion  is  both  reliable  and  relevant 
£Bef.  1]  and  £  Bef .  10]. 

(2)  Criteria  must  be  comprehensive  in  that  the  utility  of 
the  individual  being  evaluated  is  unambiguously  reflected. 

(3)  Criteria  should  possess  selectivity  and  have  ready 
applicability  £Hef.  10]. 

Finally,  since  readiness  and  criteria  are  related  to 
human  performance,  it  is  necessary  to  define  human 
performance. 

C.  BOHAH  PEBFOBflAHCE 

Much  of  the  data  used  in  human  factors  consist  of 
measures  of  some  aspect  (s)  of  human  performance.  Human 
performance  in  the  context  of  systems  often  boils  down  to 
consideration  of  how  fast  people  can  perform  their  functions 
and  how  accurately  they  can  perform  them.  How  well  functions 
are  performed  leads  directly  to  considerations  of  human 
errors  £Bef.  12]. 
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D.  THE  PROBLEM 


Thus,  the  problem  is  the  necessity  of  optimizing  the 
combat  readiness  of  HAF’s  combat  pilots,  because  of  the 
threatened  invasion  by  neighboring  countries.  So,  we  have 
to  identify  the  necessary  variables  which  will  assist  the 
planner  (commander)  to  be  certain  that  his  combat  pilots 
will  meet  the  desired  level  of  combat  effectiveness  in  terms 
of  pilot  capabilities. 

It  can  be  suggested  that  for  HAF  the  first  requirement 
is  to  continuously  optimize  combat  readiness.  To  accomplish 
this  objective  it  is  essential  that  combat  pilots  be  at  an 
acceptable  level  of  combat  readiness.  Determination  of 
combat  readiness  requires  definition  of  concept,  specifica¬ 
tion  of  criteria  and  a  definition  of  human  performance. 

In  the  chapters  to  follow  an  attempt  will  be  made  to 
define  and  discuss  the  following:  (a)  the  principles  of 
human  performance  and  (b)  the  criteria  and  their  measure¬ 
ment.  Finally,  chapters  IV  and  V  will  examine  combat  ready 
crew  performance  and  derivation  of  combat-crew  performance 
measurement  techniques. 


II-  PRINCIPLES  OF  ROMAN  PERFORMANCE 


A.  HOMAN  PERFORMANCE  REQUIREMENTS 

can  best  determine  performance  requirements  only 
after  we  have  a  good  set  of  allocated  functions.  In  the 
absence  of  function  allocation  we  will  have  a  difficulty 
identifying  and  dealing  effectively  with  human  performance 
requirements  because  we  do  not  know  what  functions  people 
will  be  expected  to  perform. 

Once  we  have  established  a  human  performance  requirement 
we  should  develop  a  way  of  measuring  it.  If  the  requirement 
relates  to  accuracy,  then  there  must  be  a  meaningful  way  to 
measure  errors.  If  the  requirement  concerns  manual 
processing  time,  there  must  be  a  meaningful  way  to  measure 
it.  For  example,  time  per  customer  contact,  number  of  items 
produced  per  hour,  and  average  keystrokes  per  day. 

Usually  the  problem  is  deciding  wnat  measure  gives  the 
best  indication  of  performance.  It  has  been  suggested  that 
with  a  little  imagination,  any  human  performance  can  be 
meaningfully  measured  [Ref.  13].  As  a  minimum,  human 
performance  requirements  should  include  statements 
concerning  errors,  manual  processing  time,  training  time 
necessary  to  ensure  the  minimum  skills,  and  job 
satisfaction.  If  we  do  not  clearly  state  the  requirements 
from  the  beginning  we  cannot  expect  human  performance 
considerations  to  be  taken  seriously.  And  when  the  system 
is  operational,  people  will  be  left  to  perform  as  best  they 
can  without  adequate  provision  for  ensuring  an  acceptable 
level  of  human  performance  [Ref.  14]. 

Other  human  performance  requirements  could  be  associated 
with  training  time  (e.g.,  total  time  to  train  clerical 


personnel  to  perform  the  basic  activities  should  not  exceed 
three  seeks).  Another  human  performance  requirement  could 
relate  to  job  satisfaction  {e.g.,  after  performing  an 
activity  for  six  months  employees  should  respond  in  a  posi¬ 
tive  way  to  their  work,  as  measured  by  a  questionnaire)  . 

The  identification  of  human  perfomance  requirements  is  a 
prerequisite  step  for  a  system's  development  process.  One 
of  the  steps  of  a  system's  development  process  is  the  task 
analysis.  Task  analysis  is  intended  to  match  the  work  to  be 
done  with  the  kinds  cf  people  who  will  do  it.  The  process 
has  four  main  parts: 

(1)  Determination  of  a  system  structure  that  gives  the 
designer  an  overall  view  or  objective  for  the  analysis. 

(2)  Identification  of  tasks  has  essentially  nine  primary 
considerations  associated  with  it.  These  include 
determining  existing  knowledge  and  skills,  deriving  skill 
level  categories,  identifying  outputs  and  inputs,  deriving 
lower  level  activities,  ensuring  that  activities  are  mutu¬ 
ally  exclusive  and  exhaustive,  and  matching  active 
complexities  with  previously  determined  skill  levels.  The 
process  may  be  repeated  any  number  of  times  until  each 
activity  is  assigned  a  single  skill  level.  Secondary 
considerations  in  the  identification  of  tasks  include 
meeting  the  system  structure  objective  just  discussed,  as 
well  as  meeting  a  full-advantage  objective.  A  designer 
attempts  to  develop  tasks  that  will  ultimately  take  full 
advantage  of  the  user  work  force.  This  is  difficult  to 
quantify,  but  during  the  analysis  process,  most  designers 
gain  a  feel  for  what  is  meant  by  taking  full  advantage  of 
the  skills  available  in  their  user  population  and  this 
should  be  reflected  as  the  tasks  are  being  identified. 

(3)  Description  of  each  task  and  organization  of  all  tasks 
into  a  flowchart  that  will  accomodate  the  variety  of 
different  transactions  a  new  system  must  accomplish. 


(4)  Identification  of  work  modules  that  synthesizes  the 
tasks  previously  identified  into  manageable  modules  of 
work.  A  human  work  module  is  a  set  of  tasKS  that  a  user 
accomplishes  as  a  part  or  all  of  his  or  her  job  [Ref.  14]. 
It  is  a  basic  unit  work.  Usually,  one  or  more  work  modules 
are  combined  to  form  a  job. 

Having  systematically  derived  work  modules  assists  in 
the  design  of  interfaces,  and  the  preparation  of  facilitator 
materials,  such  as  instructions,  performance  aids  and 
training. 


B.  FACILITATING  HOHAB  PERFORMANCE 

Once  good  work  modules  are  developed, many  types  of 
materials  can  be  developed  to  help  ensure  an  acceptable 
level  of  human  performance.  All  of  these  materials  are 
based  on  the  task  analysis  results  and  are  usually  prepared 
to  support  the  work  module  instructions. 

Once  the  designer  identifies  tasks  and  determines  work 
modules,  he  must  identify  the  specific  skills  and  knowledge 
required  for  each  work  module.  This  amounts  to  stating  his 
assumptions  of  the  precise  qualifications  of  the  person  to 
perform  the  work  module. 

As  the  work  modules  are  designed,  the  designer  should 
have  a  set  of  user  characteristics  in  mind.  The  design 
should  clearly  envision  the  person  performing  the  work. 
When  this  information  is  written  down,  it  is  called  a 
Statement  of  Minimum  Qualifications  or  SMQ  [Ref.  14]. 


The  SMQ  is  a  detailed  description  of  minimum  acceptable 
qualifications  in  terms  of  skills  and  knowledge  required  to 
efficiently  and  effectively  perform  the  work  outlined  by  the 
work  module. 

Once  the  designer  has  in  mind  the  potential  user  and  has 
designed  the  work  modules,  then  he  can  proceed  with 
developing  instructions,  performance  aids  and  training. 

It  is  vitally  important  that  human  performance 
considerations  begin  at  least  by  the  time  functions  are 
being  allocated.  Having  a  good  set  of  human  performance 
requirements  is  also  critical.  A  systematic  and  detailed 
analysis  of  the  tasks  to  be  performed  is  also  very 
important. 

If  design  considerations  are  well  done,  then  a  strong 
foundation  is  established  for  developing  interfaces  and 
facilitators,  including  Human/Computer  dialog  structures, 
instructions,  performance  aids,  training  materials. 

In  this  chapter  the  principles  of  human 
performance,  namely  (a)  Human  Performance  requirements 
and  (b)  Facilitating  human  performance  have  been  discussed. 

In  order  to  measure  one  person’s  attitude,  aptitude  and 
performance  we  may  use  criteria.  Thus,  in  the  next  chapter 
we  shall  expand  on  the  concept  of  criteria  and  their 


measurement. 


III.  CRITERIA  AND  THEIR  MEASORMMI 


As  suggested  earlier,  a  criterion  is  a  standard  which 
may  be  used  to  evaluate  a  person’s  attitude,  aptitude,  and 
performance.  Since  personnel  selection  and  training  are 
important  factors  for  HAF  to  optimize  combat  readiness  of 
its  ccmbat  pilots,  we  will  be  concerned  with  the  use  of 
criteria  for  personnel  selection  and  training  purposes. 

Selection  criteria  are  described  by  the  degree  of  corre¬ 
lation  between  selection  test  scores  and  performance 
measures  (in  real-world  situations)  ,  which  represent  the 
degree  of  a  pilot’s  success  in  performing  his  job. 

Training  criteria  refer  to  measures  utilized  in  evalu¬ 
ating  the  effectiveness  of  a  training  program  (i.e.,the 
measures  that  express  the  degree  to  which  the  attainment  of 
the  behaviorial  objectives  of  the  training  program  have  been 
met)  [Ref.  15]. 

A.  USE  CF  CRITERIA  IN  PERSONNEL  SELECTION  AND  TRAINING 

Personnel  selection  serves  to  predict  a  person's 
suitability  for  a  job,  whereas  the  purpose  of  personnel 
training  is  to  derive  a  predetermined  work  standard,  or 
other  criterion,  in  the  shortest  possible  training  time. 
Eoth  purposes  can  be  achieved  by  analyzing  and  quantifying 
the  content  and  skills  associated  with  the  job.  The  valida¬ 
tion  of  each  process  can  be  evaluated  as  follows  : 

1.  The  validity  of  a  personnel  selection  system  can  be 
evaluated  by  the  degree  of  the  relationship,  or  correlation, 
between  test  scores  against  the  criteria  measures. 

2.  The  validity  of  a  training  method  can  be  evaluated  by 
comparing  a  variety  of  different  training  methods  with  a  set 
of  the  same  criteria  for  all  cases. 


The  validity  of  selection  and  training  programs  alike  is 
affected  by  the  following  factors  : 

1.  The  degree  to  which  personnel  selection  tests  or 
personnel  training  procedures  simulate  real-world  criteria. 

2.  The  validity  of  personnel  selection  tests  and 
training  under  study. 

3.  Validity  of  the  criteria  considered,  true  and  not 
merely  correlational. 

4.  Relevance  of  the  criteria  considered  for  (a)  the  job 
studied,  and  (b)  the  test  batteries  adopted  for  selection  or 
training  purposes. 

5.  The  stage  (initial,  ultimate,  or  rate  of  learning)  at 
which  criteria  are  measured  may  affect  validities  signifi¬ 
cantly.  Typically,  criteria  measures  at  the  early  stages  of 
the  acquisition  of  skill  exhibit  higher  validity  coeffi¬ 
cients  than  those  we  find  when  the  measures  are  correlated 
with  terminal  performance  [Ref.  16]. 

6.  Because  job  performance  is  multidimensional,  taking 
only  one  criterion  as  an  index  of  job  performance  may  result 
in  artificially  high  or  low  validity  coefficients. 
Therefore,  a  multidimensional  approach  is  essential  in  the 
measurement  of  job  performance  and  criteria  utilization  in 
personnel  selection  and  training. 

7.  Evidence  indicates  that  criterion  and  test  measures 
are  typically  nonlinear  [Ref.  17]  [Ref.  18]  and  [Ref.  19], 
Since,  moreover,  assuming  their  linearity  increases  the 
error  variance  of  the  validity  coefficient,  the  validity  of 
the  assumption  must  be  tested  before  linearity  is  assumed. 

8.  Fluctuations  in  an  individual's  physiological  and 
psychological  health  condition,  need  of  achievement  motiva¬ 
tion,  and  so  on,  affect  the  validity  coefficient  of 
personnel  selection  programs  as  well  as  training  programs. 


In  summary,  it  is  hardly  possible  to  overstress  the 
importance  of  a  scund  criterion  measure  against  which 
personnel  selection  and  training  programs  can  be  evaluated 
effectively.  In  order  to  achieve  this  objective,  emphasis 
must  be  placed  at  the  outset  on  the  development  cf  such 
measures.  Without  sound  criteria  the  true  validity  of 
personnel  selection  and  training  cannot  be  assessed 
properly. 

B.  TYEES  CF  CRITERIA. 

Criteria  for  the  purposes  of  personel  selection  and 
training  lie  on  a  continuum  scale,  which  has  two  interactive 
dimensions  :  objectivity-subjectivity  and  employee-employer 
satisfaction.  Some  criteria  are  entirely  subjective;  others 
are  entirely  objective.  Most,  however,  involve  a  mixture  of 
the  two.  In  other  words  we  distinguish  two  types  of 
criteria:  (1)  Those  describing  the  satisfaction  of  the 
empoyee's  needs  and  (2)  those  describing  the  satisfaction  of 
empoyers  in  their  empcyees. 

The  ideal  criterion,  although  rarely  found  in  industry, 
is  the  one  that  reflects  the  satisfaction  of  employee's 
needs  as  well  as  that  of  empoyers'.  A  criterion  like  this 
is  used  more  frequently  for  personnel  guidance  purposes  than 
in  personnel  selection  and  training  [Ref.  20]. 

It  is  essential  to  aim  at  using  the  most  appropriate 
criterion  for  a  specific  job,  rather  than  the  most 
convenient  criterion,  but  it  is  not  always  feasible  to 
assess  the  former  or  to  obtain  data  on  it.  Thus,  the 
analyst  is  frequently  faced  with  the  practical  issue  of 
using  the  most  appropriate  criterion  which  data  are 
available  or  can  easily  be  collected. 

The  reason  for  using  a  criterion  varies  since  occupa¬ 
tional  jobs  are  different;  consequently,  a  variety  of 
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criterion  measures  have  been  developed  and  used  during  the 
last  century,  each  with  its  characteristic  weaknesses  and 
strengths.  However,  any  criterion  could  be  the  most  appro¬ 
priate  for  a  specific  job,  depending  on  circumstances  and 
purpose.  The  majority  of  the  criteria  relevant  to  today's 
occupational  circumstances  and  competence  are  listed  in 
Table  1.  • 

For  some  of  the  criteria  mentioned  in  Table  1,  objective 
data  can  be  gathered  (e.g.,  length  of  employee's  service), 
ethers  require  subjective  assessment  (e.g.,  rating  or 
ranking  by  supervisor,  subordinates,  members  of  equal 
status,  and  the  employee  himself).  For  some  criteria  (e.g., 
quality  of  output)  both  subjective  and  objective  techniques 
can  be  utilized.  Many  of  these  can  be  related  to  training 
progress  at  the  following  stages:  (a)  immediate,  at  the 
beginning  of  training;  (b)  intermediate,  at  the  end  of 
training;  (c)  ultimate,  in  real-world  operations,  after 
completion  of  training. 


CRITERIA  HEASORES 


There  are  basically  three  types  of  methods  by  means  of 
which  criteria  data  can  be  collected  : 

1.  Rating  or  ranking. 

2.  Counting  (e.g.,  the  number  of  items  produced,  the 
number  of  accidents,  the  duration  of  service) . 

3.  Establishing  work  standards  [Ref.  21]. 

In  this  chapter  we  have  seen  the  criteria  and  their 
measurement  and  specifically  (a)  Use  of  criteria  in 
personnel  selection  and  training,  (b)  Types  of  criteria  and 
(c)  Criteria  measures.  Next  we  shall  introduce  the  combat 
ready  crew  performance  measurement. 


TABLE  1 


Types  of  Criteria  Utilized  for  Personnel  Selection  S 

Training 


Criteria  reflecting  employees'  competence 
A.  Quality. of  performance 


II. 


1. 

2. 

3. 


T. 

2, 


3. 


4. 


5. 


quality  cf  output 
Spoiled  work: 

(a)  amount 

(b)  cost 
Accidents: 

(a)  number 

Jb)  cost  (financial  and  human) 
Breakages  (tools, etc.)  : 
number 


$ 

nis 

iSU 

Vari' 


,  .  cost 
Iistakes  in 
num  ber 
cost 

riability 


operation: 


B. 


6.  Variability  in  performance 
*7.  Rate  of  advancement 
*8.  Standard  trade  examinations 
*9.  Training: 

(a)  cost 

(i)  to  employer, 

{ b)  duration 
Quantity  of  performance 


(ii)  to  employees 


1.  Quantity  of  output 

2.  Earnings  on  a  commission  basis 

3.  Earned  bonus 

4.  Peak  performance 

5.  Lowest  performance 
*6.  Rate  of  advancement 

*7.  Standard  trade  examinations 
*8.  Training  : 

(a)  cost 

(i)  to  employer,  (ii)  to  employees 

(b)  duration 

Criteria  reflecting  employees'  circumstances 
Length  of  service 


4. 


III. 

1. 

2, 


Labor  turnover 
Lateness  (categorized 

(a)  numbers 

(b)  cost 
Absenteeism  : 

(a)  certified 

(i)  number,  (ii) 

(b)  uncertified 
(i)  number,  (ii) 


according  to  how  late) 


cost 

cost 


uu  f  \  a.  jl  j  w 

Criteria  reflecting  employee's  satisfaction 
from  job 

Rating  of  employee's  liking  for  his  present  jot 
Rating  of  employee's  satisfaction  with 
job  content  and  desire  for  job  enlargement 
or  job  simplification 


*  Starred  items  may  reflect  quality  and/or  quantity  cf 
performance.  Since  their  relationship  to  performance 
is  indirect,  they  are  less  valuable  and  important 
than  the  others. 
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IV.  INTRO D OCT  10 N  TO  COMBAT  READY  CREH  PERFORMANCE 

MEASOREMENT 

Recently,  the  evaluation  of  aircrew  proficiency  in 
skills  associated  with  advanced  flying  training  was  prima¬ 
rily  based  upon  the  subjective  judgement  of  instructor 
pilots.  In  addition  these  evaluations  are  supplemented  by 
the  mere  or  less *ob j ective  scores  of  gunnery  and  bombing.  In 
certain  areas,  such  as  air  combat  maneuvering  and 
combat-readiness  determination  the  evaluations  are  totally 
objective. 

Although,  efforts  have  been  made  to  measure  objectively 
the  behavioral  skills  in  the  operational  and/or  crew 
training  setting  in  an  economical  way,  these  efforts  have 
not  yet  produced  any  positive  results. 

A  great  percentage  of  the  HAF's  budget  is  absorbed  by 
the  aircrew  training  costs,  therefore,  this  area  has  become 
of  fiscal  concern.  Due  to  the  projected  increase  in  costs 
cf  operating  the  newer  weapons  systems  over  the  present 
generation  of  combat  aircraft  the  HAF’s  interest  in  the 
problem  has  intensified  in  recent  years.  Studies  to  minimize 
costs  have  shown  that  this  can  be  achieved  by  reducing  the 
training  flying  hours  and  transfer  them  (when/where  this  is 
possible)  to  lower  ccst  devices,  e. g.  simulators. 

The  HAF  is  trying  to  adapt  modern  Systems  Approach  to 
Training  techniques  to  aircrew  training  programs.  Existent 
in  the  Systems  Approach  to  Training  concept  is  student 
advancement  on  individual  proficiency  rather  than  course 
length.  The  traditional  subjective  methods  of  evaluation  may 
be  proved  insufficient  for  the  more  complex  aircrew  training 
objectives. 


A.  OBJECTIVES 


Necessity  of  improving  training  performance  information 
of  HAF's  pilots,  directed  this  study  to  the  following 
objective  based  on  experiments  and  studies  of  USA? 
[Ref.  22]  : 

j  Goal:  Systematic  definition  of  performance  measures 

appropriate  to  combat-crew  training  needs. 
Performance  measures  will  include  formal  statements  of 
methods  of  measuring  flight  crew  performance  used 
j  during  and  at  the  end  of  combat-crew  training,  and,  new 

measures  meaningful  to  combat-crew  training  and  useful 
as  tools  for  training  research. 

This  study  thus  is  an  effort  to  describe  usable  measure¬ 
ment  tools  for  utilization  in  combat-crew  training 
research. 

1 .  Instructional  System  Development. 

Research  studies  directed  toward  performance 

measurement  in  combat-crew  training  are  highly  relevant 
today  in  view  of  USAF  policy  to  employ  a  systems  approach  to 
flying  training  problems.  The  model  for  Instructional 
System  Development  (cf..  Dept,  of  Air  Force,  1970)  contains 
the  following  basic  steps: 

1.  Analyze  system  requirements. 

2.  Define  education  or  training  requirements. 

3.  Develop  objectives  and  tests. 

4.  Plan,  develop,  and  validate  instruction. 

5.  Conduct  and  evaluate  instruction. 

In  support  cf  instructional  system  development, 

measures  and  a  measurement  system  are  necessary  to: 
(1)  perform  analyses  of  systems  in  their  operational 
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environments,  (2)  establish  quantitative  instructional 
standards,  (3)  provide  an  index  of  achievement  for  each 
behavioral  objective,  and  (4)  evaluate  alternative 
instructional  content,  approaches,  and  training  devices. 

In  particular,  instructional  system  development 
requires  that  performance  standards  are  identified  so  that 
the  most  efficient  approach  is  used  to  train  the  needed 
skills  and  knowledge  to  the  desired  level  of  performance. 
Such  performance  standards  imply  performance  measurement  for 
both  the  determination  of  desirable  approaches  to  training 
and  fcr  testing  student  performance. 

All  these  events  mentioned  above  should  be 
considered  positive  and  therefore  this  model  for 
instructional  system  development  could  also  be  used  and 
adapted  by  HAF. 


this 


model 


2.  A  Measurement  System  for 
Environment. 


the  Operational 


It  will  be  necessary  to  develop  measurement  tools 
that  would  be  usable  in  the  operational  environment  of  HAF 
under  the  constraints  that  such  an  environment  implies. 
Within  the  context  of  this  requirement,  it  is  essential  that 
an  attempt  be  made  to  establish  a  list  of  parameters  to  be 
sensed,  and  the  pcint-of- view  taken  that  the  parameters 
should  be  derived  from  that  information  that  the  operational 
training  personnel  consider  to  be  meaningful  and 
significant. 

3.  An  Automated  Measurement  System. 

An  ancillary  objective  is  to  develop  a  measurement 
system  that  will  relieve  the  instructor  pilot,  to  a  maximum 
extent,  from  the  requirement  of  having  to  record  a  great 
deal  cf  information  manually  on  the  basis  that  such  activity 


degrades  his  ability  to  competently  instruct  his  student. 
This  does  not  imply  that  such  a  measurement  system  is  an 
attempt  at  automated  evaluation.  The  measurement  system 
should  certainly  include  means  for  transforming  and 
analyzing  performance  information,  but  ultimately  evaluation 
and  decision  for  training  control  is  a  human  function. 

Thus,  HAF  would  improve  and  optimize  the  quality  of 
training  of  its  combat  pilots  by  adapting  an  instructional 
system  development,  a  measurement  system  for  the  operational 
environment  and  finally  an  automated  measurement  system. 

B.  HEASOBEMENT  BASIC  ON  COMBAT-CREW  TRAINING  INFORMATION 

NEEDS 

The  strategy  employed  by  the  tJSAF  was  to  design  a 
measurement  system  that  could  acquire  that  data  identified 
as  meaningful  by  training  management  and  instructor 
personnel.  Data  were  gathered  from  sites  across  a  broad 
spectrum  of  combat-crew  training  programs  of  USAF. 

The  combat-crew  training  sites  from  which  data  were 
taken  are  listed  in  Table  2.  The  aircraft  sample  included 
heavy  (inter-  and  intra- theatre  cargo/transport,  and 
bomber)  and  high-performance  aircraft  (one-  and  twc-man 
interceptor  and  tactical  fighter) .  An  attempt  was  made  to 
(1)  consider  measurement  in  the  context  of  combat-crew 
training,  (2)  assess  measurement  already  included  as  well  as 
identify  potential  measurement  indicated  by  combat-crew 
training  personnel,  and  (3)  assess  the  constraints  placed  by 
the  environment  on  feasible,  usable  measurement  systems. 

From  the  six  types  of  aircraft  listed  in  Table  2  the  HAF 
pilots  fly  the  three  cf  them,  that  is  F-4E,  C-130E  and  A-7D. 
Thus,  the  OSAF's  attempt  and  data  found  are  of  great 
interest  for  HAF,  in  order  to  improve  training  performance 
information  for  its  ccmbat-crews. 


TABLE  2 

Data  Collection  Sites 


Place 

Castle  AFB 
Altus  AFB 
Dyes s  AFB 
Davis-Monthan  AFB 
Tyndall  AFB 
Luke  AFB 
♦George  AFB 
♦Norton  AFB 
♦♦Nellis  AFB 


Aircraft 

B- 52  F,  G  &  H 

C-141A 

C-130E 

F-4  C#  D  &  E 

F-  106  A  &  B 

A-7D 

F-4E 

C-141A 

F-4E 


*  Special  emphasis  on  crew-performance  measurement 
♦♦Special  emphasis  on  air-combat  maneuvers 
performance  measurement. 


C.  COMBAT-CHEW  PEBFGBHANCE  MEASUREMENT 

For  the  six  aircraft  mentioned  in  Table  2,  a  common 
basis  for  measurement  was  established.  This  way  it  could  be 
possible  to  apply  a  mere  or  less  modular  approach.  Allowance 
must  be  made  for  special  aircraft  characteristics;  for 
example,  the  F-106  has  no  flaps  and  the  B-52  has  a  quite 
complicated  flap  retraction  routine  compared  to  other 
aircraft.  Measurement  was  treated  for  each  of  the  following 
maneuvers; 


(1)  Takeoff  and  Climb 

(2)  Pattern,  Land  and  Go-Around 

(3)  Instruments 

(4)  Formation 

(5)  Air-Air  Intercept 


(6)  Air  Combat  Maneuvers 

(7)  Air  Refueling 

(8)  Ground  Attack 

(9)  Air  Drop  and  Air  Drop  Formation 

(10)  Radar  Navigation  and  Bombing. 


Prototype  measurement.  For  each  of  above  maneuvers,  the 
data  collected  from  the  sites  of  Table  2  were  compiled  into 
the  summary  form  shown  in  Figure  4. 1  Since  the  blanks  in  the 
summary  form  of  Figure  4. 1  indicate  needed  information,  and 
conseguently  items  fcr  measurement  development,  these  forms 
were  termed  Prototype  Measurement  since  they  form  a  model 
after  which  measurement  could  be  patterned.  Details  and 
explanations  of  a  prototype  measurement  is  presented  in 
Appendix  A. 

Measurement  Specifications.  The  parameters  which  must 
be  sensed  to  permit  measurement  are  not  immediately  evident 
from  the  information  requirements  (Prototype  Measurement)  , 
since  the  measure  specifies  the  output  of  a  computation,  and 
the  computation  itself  must  be  known  before  the  inputs  to 
the  computation  (the  parameters)  can  be  determined. 

For  example.  Figure  4. 1  indicates  that  a  measure  of 
centerline  deviations  is  desired  during  the  takeoff  roll.  It 
is  clear  that  the  distance  between  the  aicraft  position  on 
the  runway  and  the  runway  centerline  is  a  parameter  needed 
for  measurement.  The  desired  measurement  might  be  simply  the 
average  difference,  or,  conceivably,  might  involve  the  rela¬ 
tionship  between  centerline  deviation  and  heading  (or 
lateral-G,  or  brake  application)  and  thereby  indicate  the 
need  for  other  parameters  to  execute  the  calculations  for 
measurement.  Further,  the  measurement  calculations  must  be 
made  (as  indicated  in  Figure  4.1  )  from  the  application  of 
takeoff  power  until  rotation,  implying  the  need  for  ether 
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TAKEOFF  5  CL I M3* 


CONDITIONS: 

”GrosS“UFT _  Wing:  /  _  Runway:  /  ___ 

Temp.: _ “Alt .  Set. : Field  Elev.  Form  Pos.  : 


TAKEOFF  ROLL:  (To  power  until  rotation) 

”Fowef“SeI: _  _  “  “  ""Centerline  Dev. :  Min,  Max,  Av. 

Reject  Speed:  Computed  Headinq:Min,  Max,  Av. 

Time: _ *_  Dist:_  _  Bank:  S  Max,  L  Max 


ROTATION:  (Nose  gear  off  until  pitch  att.  established) 

”FcT7"3peeaT“ - STID.  Trim: - - 

Pitch:  Rate :_  _  Bank:  _ _ 

Final:”  _ ~  Centerline  Dev7T~ 

Overshoot:  Heading:  _ 3  ~ 


LIFTOFF:  (Pos.  Vert.  Vel.) 

"Airspeed:”  ”  Pitch:  Bank:  Hdg: 

vert.  Vel7”Xfter: _  Sec.:  “  “ 


GEAR-UP: 

“Gear-Up 

Pitch: 


(Handle  up  until  qear-up  S  locked) 

Speed? - vTYI  TniTT?  *  V.  77” Final: 

Bank: _  Hdg: _ 


FLAES  UP:  (Start  U£  to  full  up)  ** 
Trim:  “IF5Z  Only 

Pitch:  Bank:  fldg:  Start 

*  (IUTT)  TF“"‘* - - 

I NIT)  (F 

( INIT)  (F 


A/S 
VV  ( 
ALT 


'INAL 

FINAL 

(FINAL 


1st  POS 
2nd  Pos 
Full 


IAS  PITCH 
X  X 

x  x 

X  X 

X  X 


ALT  VV  TRIM 
X  X 
X  X 
X  X 
X  X 


CLIMB  S  LEVEL  OFF:  (De pends  on  Flight  Plan) 


Accelerate 
Climb  A/S 

Climb  MACH 
Level-off 


I  NIT  FINAL 


PWR  AZS  MACH  HDG  ALT  ALT 
“x  x”  ”  x”  “x  x”  X” 


Alt-10??  VV) 

to  Cruise)  x  x  x  x  x  x 


PITCH  TR 
x 


X 
X 

X  X 


*  Also,  mandatory  communication  &  instances  where  A/C 
limits  are  exceeded. 

**F-106  has  no  flaps 


x  x  XlH  x  x  x  x 


parameters  to  indicate  when  the  measurement  interval  starts 
and  stops. 

Since  the  pilots  of  HAF  fly  at  most  the  same  aircraft1 
as  US AF  pilots,  the  data  gathered  from  these  USAF  tases  are 
of  great  interest  for  HAF.  Of  course,  of  greatest  interest 
are  the  data  of  the  three  similar  aircraft  (F-4E,  A-7D,  and 
C-130E),  but  the  data  related  to  the  other  aircraft  should 
also  be  of  great  interest.  These  may  be  applied  to  ether 
aircraft  that  pilots  of  HAF  fly  (e.g.  F-1C  Mirage,  or 
F-5A/E) ,  or  aircraft  that  HAF  will  be  procured  in  the  near 
future. 


‘The  aircraft  that  the  HAF's  pilots  fly  and  are  similar 
to  the  USAF  aircraft  bases  from  where  data  were  gathered 
are:  F-4E,  A-7D,  and  C-130E. 
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V.  DERIVATION  OF  CCMBAT-CBEW  PERFORMANCE  MEASUREMENT. 

One  cf  the  principal  products  of  the  USAF  study  was  the 
definition  of  performance  measures  appropriate  to  combat- 
crew  training  needs.  That  study  was  based  on  interviews  with 
instructors  and  training  management  and  measurement  defini¬ 
tions  were  evolved  in  the  following  steps  : 

(1)  The  varied  requirements  posed  by  the  six  aircraft 
and  missions  were  consolidated  into  a  common  framework  which 
permitted  isolation  of  measurement  modules  (measurement 
commonality)  ; 

(2)  Discussions  of  operational  training  information 
needs  were  formalized  to  indicate  in  a  checklist  fashion  the 
measurement  development  needed  (prototype  measurement)  ; 

(3)  Measurement  parameters  and  (4)  measurement  specifi¬ 
cations  were  produced  together,  but  are  presented  separately 
for  hardware  and  software  implications,  respectively  ; 

(5)  Measurement  and  analysis  for  crew  communications 
recording  were  examined  to  provide  means  of  examining  crew 
interaction  and  individual  contributions  to  total  man- 
machine  system  performance. 


A.  MEASUREMENT  COMMONALITY 

Common  measurement  was  executed  to  permit  the  design  of 
one  simple  and  practical  measurement  system,  and  eliminate 
the  need  for  a  totally  unique  measurement  system  for  each 
aircraft.  This  measurement  system  could  apply  in  HAF. 


1 


Common  T raining  Phases. 

In  the  USAF  program  the  first  step  was  to  compare 
similar  training  phases  for  each  aircraft,  in  order  to  esti¬ 
mate  the  commonality  measurement  requirements,  as  it  is 
shown  in  Table  3.  In  cases  where  X-Y2  data  are  required  for 
measurement  it  was  found  that  these  data  could  be  obtained 
by  using  expensive  equipment  (such  as  a  multiple-target 
tracking  radar)  or  with  dif f icult-to- process  recordings, 
while  equivalent  results  could  be  obtained  with  video/photo 
sensors.  As  we  can  see  in  Table  3,  almost  all  cases  of  X-Y 
data  requirements  can  be  met  by  using  video/photo  recording. 
Those  cases  where  the  X-Y  data  cannot  be  obtained  with 
video/photo  sensors  are:  (1)  lateral  drift  across  the 
runway  during  transition,  (2)  relative  position  of  aircraft 
during  intercept  prior  to  lockon,  (3)  enroute  cross-track 
error  during  airdrop,  (4)  inflight  ranging  (out  of  sight) 
during  formation,  and  (5)  space  paths  of  multiple  aircraft 
during  air  combat  maneuvers. 

Although,  phcto-sensors  are  widely  used  by  the  HAF , 
video  is  not.  So,  the  use  of  video  films  would  be  recom¬ 
mended  because  video  is  a  modern  technology  equipment  with 
many  advantages  (e.g.,  film  does  not  need  development  and 
therefore  can  be  used  immediately  after  flying  during 
debriefing) . 

Not  all  maneuvers  were  taught  at  all  sites  from 
which  data  was  taken.  For  example,  the  operational  C-130 
squadron  did  not  explicity  train  transition  maneuvers; 
nevertheless,  competent  information  was  obtained  for 
measurement.  Since  all  combat  maneuvers  were  not  taught  at 
the  combat-crew  training  squadrons,  they  had  gaps  in  data 


2The  X-Y  data  are  not  in  the  form  of  a  tabulation  of  X- 
and  Y-values,  but  they  are  position  information  such  as  the 
relationship  between  tanker  and  refueling  aircraft  from  the 
tanker  lights. 


TABLE  3 

Reguirements  For  X-T  Data 


Phase 


Parameter 


Obtainable 

vITE - 

Vldeo/Photo ? 


1 

2 

4 

t; 

6 

7 

8 
9 

10 
1  1 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 


Transition 

Intercept 
(Prior  to  Lockon) 

Air  Refueling 


Air  Drop 

Formation 
Ground  Attack 

Dart  Firing 
Air  Combat 


Ground  Track 

Centerline  Dev 

Lat.  Drift 

Threshold 

Dist.  Down  RNWY 

Spacing 

TGT.  Azimuth 

TGT.  Elevation 

TGT.  Range 

TGT.  Range  Rate 

TGT.  Aspect  Angle 

Tanker  Range 

Tanker  Range  Rate 

Centerline  Displ. 

Lights  Up 

Lights  Down 

Lights  Fore 

Lights  Aft 

Altitude  Error 

Cross  Track  Error 

Position  Error 

Range  from  Lead 

Bearing  from  Lead 

A  Altitude  from  Lead 

Actual  Air  Release  Pt. 

Range 

Range  Rate 
Bearing 

TGT.  Slant  Range 

Aim  Point  Error 

Bomb  Fall  Line 

Flight  Path 

Spacing 

Range 

Azimouth 

Elevation 

TGT.  Range 

TGT.  Range  Rate 

TGT.  Aspect  Angle 

TGT.  HDg  Cross  Angle 

Elevation 

Space  Path 


RA* 

RA 

No 

RA 

RA 

RA 

NO 

No 

No 

NO 

No 

RA 

RA 

Yes 

Yes** 

Yes** 

Yes** 

Yes** 

RA 

No 

Yes 

RA 


RA 

RA 

Yes 

RA 

RA 

RA 

RA 

Yes 

Yes 

RA 

RA 

RA 

RA 

Yes 

RA 

RA 


RA 

No 

RA 

No 


*  RA=Feduced  Accuracy 
♦♦Obtainable  with  Viaeo/Photo  System, 
tut  not  easily  otherwise. 
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collection.  They  tried  to  fill  these  gaps  by  cross-checking, 
where  possible,  with  ether  aircraft  training. 


The  following  are  observations  of  the  degree  of 
commonality  of  training  phases  for  the  six  USAF  aircraft  and 
HAF's  aircraft  also: 

(1)  TRANSITION  (TR)  .  Transition  is  a  phase  of 
combat-crew  training.  Transition  maneuvers  appear  in  a 
common  fashion,  but  the  manner  in  which  they  are  performed 
is  significantly  different  for  different  aircraft.  This 
phase  has  teen  already  adapted  by  HAF. 

(2)  INSTRUMENTS  (INST)  .  Instruments  is  a  common 
flight  phase  for  all  aircraft  (also  adapted  by  HAF)  and 
conseguently  common  measurement  (criteria)  is  conceptually 
possible. 

(3)  FORMATION  (FORM).  Formation  flight  is  performed 
by  all  six  aircraft  as  a  means  to  optimally  employ  composite 
flight  and  provide  individual-ship  effectiveness.  It  is 
considered  as  a  common  flight  phase  among  all  aircraft, 
however,  a  number  of  types  of  formation  exist  for 
specialized  missions,  each  appropriate  only  to  specific 
aircraft.  The  formation  phase  is  widely  used  in  HAF  by  high 
performance  aircraft  (as  in  F-4E,  A-7D,  F-1C,  F-5A/B)  and  is 
very  limited  among  multi-engine  aircraft  (C-130,  DC-6, 
Albatros).  Measurement  differences  will  occur  between  the 
categories  of  high  performance  aircraft  and  multi-engine 
heavy  aircraft. 

(4)  AIR-AIR  INTERCEPT  (AA)  .  Air-to-air  intercept 
and  weapons  delivery  utilizing  airborne  radar  is  accom¬ 
plished  with  only  the  F-4  and  F-1G6  aircraft  (in  HAF  with 
F-4E  and  F- 1C) .  While  the  F-4,  F-1C  and  F-106  maneuvers  and 
equipment  differ,  the  same  basic  measurement  requirements 
are  presented.  Almost  the  same  basic  measurement  require¬ 
ments  could  also  apply  in  F-5A/3  of  HAF,  against  the  event 
that  air-air  intercept  is  accomplished  by  ground  radar. 
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(5)  BASIC  FLIGHT  MANEUVERS  (3FM)  AND  AIR  CCilEAT 
MANEUVERS  (ACM) .  EFM  and  ACM  are  grouped  together  in  F- 4 
training,  and  could  be  grouped  also  in  F-5A/3  and  F- 1C  of 
HAF,  while  5FM  and  Formation  are  grouped  together  in  A- 7 
training.  The  BFM/ACM  grouping  was  maintained  for  purposes 
of  measurement  problem. 

(6)  AIR  REFUELING  (AR)  .  Air  refueling  can  cccur 
with  fcur  of  the  six  USAF  aircraft  (B-52,  F-106,  F-4,  A-7), 
tut  is  cnly  considered  a  difficult  maneuver  for  the  E-52. 
In  HAF  air  refueling  can  occur  only  in  F-4E  and  A-7  but  is 
not  used  since  there  are  not  any  tankers  available. 

(7)  GROUND  ATTACK  (GA)  AND  DROP.  A  number  of 
training  phases  are  devoted  to  F-4  and  A-7  (both  aircraft 
are  available  by  HAF)  ground  attack  (Day,  Tactical,  Night, 
with  various  weapons  and  delivery  modes) ,  but  common 
measurement  was  judged  to  be  appropriate. 

Air  Drop  (combat  airlift  mission)  maneuvers  for 
C-141  and  C-130  training  perhaps  superficially  resemble 
level-flight  ground  attack,  but  guite  different  measurements 
are  posed.  Ground  attack  and  drop  phases  have  been  also 
adapted  by  HAF  but  in  this  casethe  measurement  is  different. 

(8)  RADAR  NAVIGATION  AND  EOMBING  (RNB)  .  Navigation 
by  use  of  radar,  and  subsequent  delivery  on  target  of  either 
ordnance  or  cargo,  occurs  with  most  of  the  aircraft  of  the 
sample  except  the  F-106,  and  compatible  mission  performance 
measurement  is  indicated.  In  HAF  this  phase  is  used  by  all 
aircraft  except  these  that  are  not  equipped  with  radar 
(e.g.  ,  F-5A/B)  . 

2.  Summary  of  T raining  Phases. 

The  examination  of  the  USAF  study  shows  that 
measurement  (criteria) ,  for  the  training  phases  of  the 
sample  of  six  USAF  aircraft,  are  composed  as  follows  : 

Transition 

Instruments 


Formati on 

Air-to-air  Intercept 
Air  Combat 
Air  Refueling 
Ground  Attack 
Air  Drop 

Radar  Navigation  and  Bombing 

All  of  these  training  phases  {except  air  refueling) 
are  widely  used  by  the  aircraft  of  HAF.  From  both  studies, 
DSAF’s  and  present,  it  is  concluded  that  not  all  of  the 
training  phases  are  applicable  to  all  types  of  aircraft.  For 
this  reason,  it  is  required  to  ignore  the  measurement 
developed  in  cases  where  a  specific  phase  could  not  be 
applied . 

3 .  Common  Measurement  for  Maneuvers 

Each  phase  of  flight,  tentatively  considered  to 
permit  common  measurement , was  examined  for  detailed  measure¬ 
ment  requirements.  All  needed  mesurements,  for  each  maneuver 
used  in  the  USAF  study,  were  provided  by  (1)  interview  notes 
with  Instructors  and  Training  Managers,  (2)  Aircraft 
Techical  Orders,  Dash-one  flight  manuals  for  each  aircraft, 
(3)  Phase  Manuals,  (4)  Instructor  Guides,  and  (5)  other 
specialized  documents.  Important  information,  as  judging 
factors  and  common  errors  were  noted  for  each  maneuver, 
along  with  other  important  remarks  that  an  instructor  pilot, 
or  training  manager,  would  consider  and  thus  translated  into 
objective  measurement. 

For  example,  measurement  requirements  were  noted  for 
each  aircraft  during  takeoff  and  climo-out  maneuvers  (Runway 
Roll,  Rotation,  liftoff.  Gear-up,  Flaps-up,  Climb  and 
level-off).  A  matrix  of  measurement  requirements  was  thus 
produced,  allowing  comparison  across  aircraft.  It  was  noted 
that  takeoff  and  climt-out  are  essentially  the  same  for  all 
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aircraft,  with  the  exception  of  the  importance  of  some 
maneuvers  {e.g.#  rotation  is  critical  with  the  F-10€),  vari¬ 
ations  in  aircraft  design  (e.g.,  F-106  has  no  flaps,  B-52 
involves  a  complicated  flap  schedule),  and  climfc  profiles 
depend  on  the  specific  mission  and  clearance. 

Although  there  were  some  differences,  in  measurement 
requirements  among  different  types  of  aircraft,  the 
similarities  proved  to  be  a  lot  more.  Also,  a  modular 
approach  to  measurement  was  suggested,  that  is,  measurement 
could  be  produced  by  examining  every  maneuver  in  sequence 
for  every  type  of  aircraft. 

All  of  the  above  mentioned  maneuvers  are  used  by  the 
Hellenic  aircraft  and  thus  common  measurement  for  maneuvers 
in  a  modular  approach  could  also  be  considered  by  all 
aircraft. 

B.  PHOTOTYPE  MEASUREMENT. 

After  the  examination  of  measurement  commonality  of 
training  phases  and  maneuvers,  the  USAF  study  developed,  in 
the  form  of  formatted  measurement  outputs,  examples  of  the 
required  information  for  training.  That  is,  if  a  measure  of 
centerline  deviation  was  indicated  to  be  desirable,  this 
would  be  noted.  In  this  way  they  recorded  all  of  the  known 
information  requirements  for  a  given  phase  of  flight,  and 
then,  they  formed  the  data  found  into  a  format  to  resemble 
measurement  output.  This  output  is  termed  here  as  Prototype 
Measurement. 

An  example  of  prototype  measurement  is  displayed  in 
Figure  4.1  for  takeoff  and  climb.  Details  and  explanations 
of  this  (prototype  measurement)  example  is  presented  in 
Appendix  A.  A  blank  or  x  in  Figure  4.1  indicates  one  or 
more  numerical  entries  to  be  determined  as  a  result  of 
measurement.  For  every  one  of  the  training  phases  similar 
measurement  was  developed. 
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In  the  USAF  study  the  prototype  measurement  was  produced 
for  the  following  trainig  phases: 

(1)  TRANSITION.  For  convenience,  transition  was  divided 
into  (a)  Takeoff  and  Climb,  and  (b)  Pattern,  Land  or 
Go-arcund.  As  previously  noted,  measurement  appropriate  to 
the  composite  of  six  aircraft  is  indicated;  the  F-106  and 
B-52  require  special  treatment. 

(2)  INSTRUMENTS.  Measurement  for  instrument  flight  is 
treated  as  the  sum  of  (a)  basic  aircraft  control 
performance,  and  (b)  navigation  performance  with  respect  to 
air  traffic  control  requirements.  While  required  measure¬ 
ment  modules  for  each  of  these  classes  of  measures  can  be 
specified,  it  is  difficult  to  present  detailed  measurement 
except  for  specific  clearances  and  published  procedures. 

(3)  FORMATION.  Measurement  for  formation  flight  is 
super-imposed  onto  mission  performance  measurement,  thus 
making  formation  measurement  difficult  to  isolate.  In 
particular,  tactical  formation  performance  was  not  clearly 
identified  in  this  study  in  an  objective  quantitative 
fashion.  It  may  be  necessary  to  rely  heavily  on  instructor 
subjective  measurement  (which  may  be  quite  satisfactory  when 
the  instructor  is  in  a  position  to  observe  performance  ) , 
except  for  measurement  associated  with  join-up,  close  forma¬ 
tion,  and  in-trail  formation. 

(4)  INTERCEPT.  In  order  to  be  specific,  intercept 
measurement  was  based  primarily  on  the  F-106,  however,  the 
intercept  problem  is  essentially  the  same  as  the  F-4.  Of 
cource,  a  two-man  crew  performance  is  expected  to  be  better 
in  the  F-4.  There  is  a  radar  observer  to  perform  the  scope 
work  and  differences  in  the  equipment  and  capability  suggest 
that  slightly  different  strategies  might  be  employed. 


(5)  A  IF.  COMEAT  MANEUVERS.  Prototype  air  combat  maneu¬ 
vering  measurement  dealt  primarily  with  set-ups  during 
initial  training  and  for  dart3  firing. 

(a)  Air  combat  set-ups  involve  placing  attacking  and 
defending  aircraft  in  fixed  initial  positions,  then  freeing 
them  to  perform  a  maneuver,  and  subsequently  judging  from 
the  final  position  wnether  the  maneuver  was  properly 
performed  and  whether  proper  advantage  of  the  tactical  situ¬ 
ation  was  taken.  Thus,  measurement  can  be  directed  to 
description  of  the  maneuvers  performed  (e.g.,  hard  turn, 
hi-lo-speed  yo-yo,  scissors,  barrel  roll,  etc.) ,  and  to 
determining  whether  a  given  student  was  able  to  improve  his 
situation. 

(b)  For  dart  firing  the  prototype  measurement  assumes  a 
butterfly  pattern  or  the  eguivalent.  A  pass  is  made  over  the 
target,  a  time  hack  is  taken  crossing  the  dart,  the  pilot 
must  circle  back  to  make  an  intercept  to  put  a  hole  within 
the  target  in  a  given  amount  of  time.  Thus,  the  time  and 
hits  on  each  pass  is  measured;  additionally  the  range, 
azimuth,  and  elevation  at  the  beginning  and  end  of  firing 
describe  the  firing  position.  Fouls  are  called  for  low 
airspeed  and  for  firing  within  a  minimum  firing  range. 

(6)  AIR  REFUELING.  As  Air  refueling  is  especially  diffi¬ 
cult  in  E-52  combat-crew  training,  the  prototype  measurement 
was  tailored  to  the  E-52  tasks  and  to  Strategic  Air  Command 
reguirements. 

(7)  GROUND  ATTACK.  During  training,  ground  attack  is 
divided  into  ground  attack,  ground  attack  night,  ground 
attack  tactical.  Ground  attack  measurement  was  dictated  by 
information  needed  for  standard  error  analysis  of  weapons 
delivery  accuracy  and  by  ground  attack  procedures.  Some  of 


3The  dart  is  a  kind  of  air  to  air  target,  towed  by  a  tow 
aircraft. 


the  measures  are  specially  designed  to  apply  to  the  A-7d 
heads-up  display. 

(8)  AIR  DROP.  Extensive  prototype  measurement  was 
dictated  for  the  Comlat  Airlift  Mission  since  very  detailed 
procedures  are  adhered  to  throughout  the  many  portions  of 
the  mission.  Crew  performance  is  especially  important  in 
this  mission. 

(9)  RADAR  NAVIGATION  AND  BOMBING.  Prototype  measurement 
for  Radar  Navigation  and  3ombing  is  heavily  dependent  on  the 
characteristics  of  the  avionics  used.  As  only  the  B-52  is 
equipped  with  low-level  terrain  avoidance  radar  (of  the 
aircraft  sampled  in  this  study),  measurement  for  these 
maneuvers  has  been  tailored  to  this  application. 

Of  course,  from  the  examination  of  the  prototype 
measurement  we  can  see  that  the  measurement  requirements  are 
very  extensive  and  complex.  For  example,  it  may  be  seen  that 
to  describe  just  Takeoff  and  Climb,  it  is  necessary  to 
compute  50-100  numbers.  Subsequently,  if  full  mission 
measurement  is  reguired,  including  transition,  instruments, 
formation,  and  weapons  delivery,  it  will  require  a  very 
large  set  of  descriptive  numbers.  But  probably,  this  detail 
is  very  necessary  to  support  the*  training  process.  The 
j  instructor  may  need  considerable  detail  to  perform  his  job 

well.  As  a  consequence,  the  prototype  measurement  could  be 
very  useful  for  HAF,  since  today  the  evaluation  of  aircrew 
proficiency  in  skills  associated  with  advanced  flying 
l  training  is  primarily  based  upon  the  subjective  judgement  of 

instructor  pilots. 

C.  MEASUREMENT  PARAMETERS. 


Measurement  is  the  process  of  producing  measures  which 
are  indices  of  performance  such  as  the  conditions  existing 
at  the  time  of  weapon  release  during  ground  attack. 


deviations  from  the  clearance  during  IFR  flight,  ar.d  flight 
conditions  at  lift  off.  The  measures  are  commonly  computed 
from  flight  variables  (e.g.,  altitude,  airspeed,  heading, 
etc.)  and  other  raw  information  which  must  be  recorded  at 
some  time;  these  are  termed  measurement  parameters.  The 
measurement  parameters  must  be  specified  so  that  the 
required  sensors  and  recording  equipment  can  be  determined. 
However,  since  the  measures  are  the  result  of  a  computation, 
the  details  of  the  computation  must  be  known  so  that  the 
inputs  to  the  computation  (the  parameters)  can  be 
established. 

Figure  5.1  depicts  the  relationship  between  the  speci¬ 
fied  measures,  the  computation,  and  the  measurement  parame¬ 
ters.  The  corresponding  data  processing  is  shown  in  Figure 
5.2.  It  is  assumed  that  the  flight  maneuvers  will  be 
divided  into  segments,  so  that  different  measures  may  be 
computed  as  appropriate  for  each  segment  (for  example, 
different  measures  are  required  during  takeoff  roll  than 
during  climb-  out) .  Consequently,  it  may  be  seen  that  the 
method  of  determining  when  to  start  and  stop  the  computation 
of  a  specific  measure  may  require  the  -  recording  of  measure¬ 
ment  parameters  in  addition  to  those  required  during  measure 
computation  (e.g.,  the  recording  of  weight-of f-the- wheels  to 
indicate  liftoff). 
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Figure  5.2  Example  Raw  Data  Processing 
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In  Figure  5.  1,  the  output  measures  (0)  correspond  to  the 
information  requirements  symbolized  by  the  blanks  in  the 
prototype  measurement  forms;  that  is,  if  the  prototype 
measurement  forms  indicated  that  a  measure  of  centerline 
deviation  is  needed,  then  parameters  must  be  recorded  and 
computations  developed  which  will  answer  the  required 
measure,  the  following  types  of  parameters  may  be  needed  in 
addition  to  the  basic  test  parameters  (M)  (for  example  just 
mentioned,  the  basic  test  parameter  would  be  a  recording  of 
the  deviation  from  the  runway  centerline  during  takeoff 
roll) ;  (1)  parameters  for  implementing  logic  to  start  and 
stop  measurement  computations  (S)  ,  (2)  information  related 
to  desired  performance  (D),and  (3)  error  information  derived 
from  the  differences  between  actual  and  desired  performance 
(E)  .  In  short,  given  output  measures  (0)  ,  to  determine 
other  parameters  which  must  be  sensed  (M,S,D,E),  it  is 
necessary  to  determine  logic  and  computations  to  be  used  in 
measurement  data  processing  (i.e.,  the  measurement 
algorithms)  . 

Assuming  automated  measurement,  i.e.,  parameters  are 
automatically  recorded  for  subsequent  computer  analysis,  the 
primary  details  of  measurement  computation  (Figure  5.2  shows 
a  representative  flow  diagram)  are  presented  in  Figure  5.3 
for  each  maneuver  and  maneuver  segment  of  combat-crew 
training  phases.  The  figure  5.3  indicates  the  name  of  each 
measure,  the  specific  function  to  be  computed,  and  the 
start/stop  conditions  for  controlling  computation.  For 
example,  centerline  deviation  during  the  takeoff  roll  is 
desired  output  information,  the  average,  minimum  and  maximum 
deviation  are  the  specific  computations  which  should  be 
performed  between  brake  release  and  rotation.  Comments  are 
also  provided  as  considered  appropriate  by  the  analyst  to 
point  up  alternatives,  or  where  problems  may  be  encountered 
during  design. 
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Since  figure  5.3  indicates  the  functions  to  be  computed, 
the  conditions  under  which  computation  should  occur,  and 
indirectly  the  source  information  upon  which  to  base  compu¬ 
tation,  the  basic  information  is  provided  to  allow  prepara¬ 
tion  of  computer  programs  for  automatic  measurement. 

When  a  list  of  parameters  was  developed  to  show  all  the 
required  sensors  and  recording  needed  for  total  measurement, 
it  was  seen  that  the  resulting  parameters  could  be  placed  in 
the  following  overlapping  categories:  (1)  pictorial  infor¬ 
mation  (e.g.,  out-  the-window  radar),  (2)  analog  information 
(e.  g.,  time-varying  quantitative,  such  as  airspeed), 
(3)  discrete  information  (e.g.,  weapon  release),  (4)  audio 
information  (e.g.,  communications),  and  (5)  desired 
performance  and  existing  conditions.  These  parameters  were 
later  assigned  to  alternative  devices  for  data  acquisition. 
After  tradeoff  analyses  were  conducted,  a  hybrid  audio¬ 
video/photo  -digital  recording  system  was  adopted.  Although 
many  parameters  could  be  acquired  by  either  video-photo  or 
digital  recording  devices  (to  allow  a  partial  system  to  have 
a  stand-alone  capability) ,  tentative  parameter  allocations 
are  listed  in  Tables  4  and  .5.  Audio  recording  will  be 
accomplished  with  either  video-photo  or  digital  recording 
devices.  Desired  performance  and  conditions  are  manually 
derived  from  briefing/debriefing  sessions  and  documents. 
Additionally,  spatial  coordinates  (X-T  data)  may  be  obtained 
if  data  are  collected  on  an  instrumented  range  including 
tracking-radar  equipment,  although  equivalent  information 
may  be  available  from  video-photo  recording  as  shown  in 
Table  3. 

After  detailed  trade-off  analyses,  the  use  of  video  or 
photo  techniques  was  emphasized  due  to  lower  costs,  flexi¬ 
bility  of  application  and  simpler  development  compared  to 
other  all-electronic  techniques;  however,  the  problems  asso¬ 
ciated  with  cockpit  installation  and  an  unfortunate  tendency 


TABLE  4 

Parameters  for  Digital  Data  Acquisition 


TIFIAAGDA  A 

RNONIIRAI  C 

ASRTRRORR  C 

N  T  M  E  U  T  U 

SRASRDN  C  R 
ICJTCERDFO  A 

TMIEFO  I  M  C 
IEOPUPARB  Y 

0  N  N  T  E  TEA 

NT  L  T  T 

S  A 

C 
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AIRCRAFT  PARAMETERS 

1.  Pitch  (Pitch  flte) 

2.  Roll 

3.  Heading 

4.  Airspeed 

5.  MACH 

6.  Altitude 

7.  Vert.  Vel. 

8.  Angle  of  Attack 

9.  Accelaration  (G)  , 

10.  Power  (RPM,  EPR, 
TIT,  Fuel,  Flow) 

11.  Fuel  Quantity 


XX  X  X  X  X  X 

XX  X  X  X  X  X 

xxxx  xxxx 

XXX  xxxxx 
XX  X 

XX  xxxxxx 

XX  xxxxxx 

XXX  XX 

X  X  XX 

XX  xxxxxx 

XX  X 


CCNTROL  PARAMETERS 
“l~3TIcirTPrTcTir~ 

2.  Stick  (Roll) 

3.  Rudder 

4.  Flap  Position  x 

5.  Stan  Trim  Position  x 


x  x 

X  X 

X 

X  X 


X 

X 

X 

X 


±1  degree 
±1  degree 
±1  degree 
±1  knot 
±.02  MACH 
±10  feet 
±50  fpm 
±1  unit 
±.  5g 


±1%  Full  Scale 
±5%  Full  Scale 


±5%  Full  Scale 
±5%  Full  Scale 
±5%  Full  Scale 
±5??  Full  Scale 
±5"  Full  Scale 


BINARY  DISCRETE  PARAMETERS 
""TTTErust  Reverse"  x 

2.  Speed  Brakes  x  x  x 

3,4. Main,  Nose  Gear 
Contact  x 

5.  Nose  Steer  Engaged  x 

6.  Gear  Select  x 

7.  Drag  Chute  x 

8.  Hteel  Brakes  x 

9.10.  Red,  Green  light 
1i.  Weapon  Release 

f Pickle)  x 

12,16.  Crewmember 
Voice  Switch  xxxxx 

17,19.  Marker  Beacon  x 

20,24.  Event  Marker  xxxxx 


x 

x 

XXX 

xxxx 

xxxx 


xxxxxxxxx  Hrs,  Min,  Sec, 

1/100  Sec. 


TIME 

1.  GMT  (Range  Time) 


for  loss  of  data  should  he  noted,  but  it  is  believed  neces¬ 
sary  to  accept  these  deficiencies  for  low-cost  combat-crew 
training  application. 

Accuracy  requirements  for  measurement  parameter  sensing 
and  recording  are  listed  at  the  right-hand  side  of  Table  4. 
These  accuracies  are  referenced  to  the  information  displayed 
to  the  crew  (for  example,  the  required  airspeed  tolerance  is 
plus  or  minus  of  the  deviation  between  recorded  values 
and  those  displayed  to  the  pilot  on  his  airspeed  indicator) 
since  the  criterion  given  the  crew  is  that  they  maintain 
vehicle  parameters  within  specified  tolerances  referenced  to 
their  instruments  (typically  5-10  knots  for  airspeed 
control) .  The  tolerances  listed  in  Table  4  are  approxi¬ 
mately  1/10th  the  tolerances  required  of  the  crew. 

As  a  result,  (a)  if  HAF  has  to  make  a  choice  between 
video/photo  and  digital  recording  approaches  to  measurement 
system  design,  then  the  video/photo  recording  would  be 
chosen  for  cost,  information  provided,  flexibility  and  ease 
of  use.  (b)  A  hybrid  system  combining  the  advantages  of 
both  is  preferable  to  either  type  of  recording  alone.  Due 
primarily  to  cost,  the  bulk  of  measurement  parameters  would 
be  derived  from  a  video/photo  system,  and  the  remainder  with 
a  small  digital  recording  capability. 

D.  HEASOEEHENT  DESCBIPTIONS 


The  gross  operations  involved  in  measurement  computation 
are  presented  in  the  flow  diagram  in  Figure  5.2  Each 
parameter  must  be  sampled  (at  a  sampling  rate  of  2,  10  or  20 
times  a  second,  depending  on  the  application)  and  tested  to 
determine  if  conditions  are  appropriate  to  start  measurement 
computation,  and  later  to  stop  computaton.  During  the 
measurement  interval  (or  at  specific  conditions,  e.  g. , 
flaps-up),  one  of  the  following  statistics  is  calculated: 


TABLE  5 

Parameters  for  Video/Photo  Data  Acguisition 

PHASE 

PARAMETERS 

Transition 

Runway  Centerline  Deviation,  Lateral 

Drift,  Threshold  Crossing,  Distance  Down 
Runway,  Ground  Track. 

Instruments 

TACAN  {Frequ.,  Course  Set,  Course  Error, 
Bearing,  DME) : 

VOR  {Frequ.,  Course  Set,  Course  Error, 
Bearing)  ; 

ILS  {Frequ.,  Localizer  Error,  Glide  Slope 
Error,  Marker  Beacon) . 

Intercept 

Target  use  of  ECM .  Maneuvering 

Radar:  Azimuth.  Elevation,  Range,  Range 
Rate,  Range  Gate,  Steering  Dot  Error, 
Firing  Circle  Radius,  Lockon  Pulse. 

IF  Gam,  Video  Gain,  Erase  Intensity. 

Refueling 

Range  to  Tanker.  Range  Rate,  Probe 
Engagement,  Centerline  Displacement, 

Lights  (Op,  Down,  Fore,  Aft) ,  Attitude 
Error. 

Air  Drop 

Crcsstrack  Error,  Groundspeed,  Terrain 
Clearance,  Ranqe/Bearing/ AAltitude  from 
Lead  A/C,  Red/Green  Drop  Lights,  Actual 
Air  Release  Point. 

Formation 

Spacing:  Range,  Range  Rate,  Bearing, 
AAltitude. 

Ground  Attack 

Target  Slant  Range,  Aim  Point  Error,  Eomb 
Fall  Line,  Flight  Path  Error,  Spacing 
in  Range  Pattern. 

Dart  Firing 

Range,  Azimuth,  Elevation,  Hits. 

air  Combat 
Maneuvers 


Target  Range,  Range  Rate,  Aspect  Angle, 
Heading  Crossing  Angle,  Elevation. 


(1)  the  value  of  a  parameter,  (2)  minimum  value,  (3)  maximum 
value,  (4)  mean,  or  (5)  standard  deviation.  Thus,  measure¬ 
ment  is  defined  by  specifying  start/stop  conditions  and  one 
of  the  five  statistics. 

An  example  of  a  measurement  specif ication  patterned  in 
this  fashion  is  shown  in  Figure  5.3.  Measurement  specifica¬ 
tions  were  produced  for  each  common  training  phase. 

While  these  specifications  initially  assume  automatic 
recording  and  computation,  they  can  also  be  used  to  describe 
manual  data  processing  (e.g.,  a  discrete  signal  indicating 
wheels-up) .  These  parameters  can  also  be  used  to  start  and 
stop  manual  processing  such  as  scanning  for  out-of-tclerance 
conditicns.  During  the  measurement  interval,  the  value, 
minimum  or  maximum  of  a  parameter  may  be  determined  manu¬ 
ally,  but  manual  processing  for  computation  of  a  mean  or 
standard  deviation  is  judged  to  be  excessively  laborious  and 
time-consuming  since  a  large  number  of  data  samples  (at  2  or 
more  times  a  second)  is  necessary.  Consequently,  the 
measurement  specifications,  as  exemplified  in  Figure  5.3, 
are  suitable  for  (1)  defining  software  for  digital  computer 
measurement  processing,  or  (2)  manual  processing  procedures. 

Today,  HAF  specifies  its  measurement  specifications 
mainly  by  manual  processing  procedures.  Lately,  HAF  has  been 
interested  in  computer  measurement  processing.  Thus,  HAF 
should  consider  the  measurement  specifications  mention  id  in 
this  section. 

£.  CCH MON ICATION  ANALYSIS 

Measurement  for  crew  performance  overlays  the  system 
performance  measurement  thus  far  discussed.  The  performance 
demonstrated  in  A-7  or  F-106  aircraft  clearly  involves  only 
one  man;  however,  the  same  mission  may  be  flown  by  two  men 
in  the  F-4  aircraft,  requiring  additional  measurement  to 
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investigate  crew  interaction  and  diagnose  individual 
performance  contributions.  The  combat  airlift  mission  in 
the  C-130  and  C-141  aircraft  involves  such  close  coordina¬ 
tion  of  pilots,  navigators,  loadmasters  and  engineers  that 
it  is  difficult  to  isolate  an  individual’s  performance  (even 
the  crewmembers  themselves  cannot  be  sure  of  the  adequacy  of 
their  performance) . 

An  individual's  performance  can  be  assessed  by  relating 
measurement  at  his  workstation  to  overall  system 
performance;  this  type  of  measurement  is  subsumed  under  the 
previous  measurement  discussion.  However,  the  interaction 

between  crewmembers  requires  analysis  of  communications - 

presenting  somewhat  different  requirements  than  system 
performance  measurement.  The  following  paragraphs  present 
important  topics  related  to  crew  performance  measurement. 

1 .  Communication  Measurement  Categories. 

Communications  measurement  must  treat  at  least  two 
gross  types  of  crew  interaction:  (1)  information  is 
exchanged  to  aid  another  crewmember  in  performing  his  duties 
(e. g.,  when  the  F-4  Weapon  System  Officer  acts  as  a  good 
',copilot,,)  ,  and  (2)  a  crewmember  provides  a  directive  role 
in  guiding  another's  performance  (e.g.,  when  the  F-4  Weapon 
System  officer  provides  directive  commentary  to  the  Aircraft 
Commander  in  air-air  intercept)  .  In  the  later  case  direct 
links  between  auditory  commands  and  system  performance  can 
be  identified,  allowing  communication  to  be  measured  in 
terms  of  resulting  performance  changes. 

Six  categories  of  measurement  related  to  information 
transfer  were  examined: 

(J.)  Timing .  Measures  of  information  timing  should 
relate  to  (a) jammimg  more  important  messages,  (b)  providing 
information  at  the  wrong  time,  (c)  delay  in  providing 


information,  and  (d)  providing  information  at  a  rat€  not 
permitting  effective  response  by  another  crewmember. 

(2)  Accuracy.  Measures  of  accuracy  require 
comparison  of  what  is  said  in  relation  to  the  measured  situ¬ 
ation  (e.g.,  was  altitute  reported  correctly?). 

(3)  Brevity.  As  radio  and  interphone  traffic  often 
exceed  channel  capacity  in  combat,  measurement  should 
address  communication  duration  and  comparison  brevity  code. 

(4)  Number  and  Frequency.  Also  in  relation  to 
communication  brevity,  the  number  of  communications  and 
frequency  cf  communications  can  be  measured. 

(5)  Informa tion  content.  Measures  of  information 
transmitted  per  unit  time  are  quite  important  although 
usually  practically  difficult  to  obtain.  Other  measures  such 
as  time,  number  and  freguency  are  often  confounded  since  a 
crewmember  may  convey  much  information  in  a  short  time  or 
few  transmissions,  while  another  may  say  little  in  a  long 
time  or  many  transmissions;  without  knowledge  of  the  infor¬ 
mation  content  it  would  be  difficult  to  evaluate  these 
situations. 

(6)  Performance  changes.  The  performance  changes  of 
the  vehicle,  desired  as  a  result  of  communicating,  define 
measurement  in  terms  of  links  between  auditory  data  and 
system/mission  performance  data.  For  example,  turning 
performance  can  be  measured  following  a  "hard-as-possibleM 
direction  to  the  pilot  of  an  F-4  aircraft. 

2.  Auditory  Data  Processing. 

A  computer-assisted  manual  auditory  processing 
system  is  required  since  automatic  voice  decoding  equipment 
is  not  available.  Expert  personnel  are  therefore  required  to 
identify  complex  performance  and  to  structure  processing 
rules  for  data  clerks  who  reduce  data  to  a  form  allowing 


input  into  a  computer. 


Ihree  auditory  data  processing  problems  imposed 
requirement s  for  measurement  system  design: 

a.  Identification. 

In  spite  of  normal  communications  clutter,  it  is  necessary 
to  clearly  identify  who  is  talking,  even  if  two  transmis¬ 
sions  are  simultaneously  made  (jamming) .  Voice  key 
circuitry  is  recommended  to  provide  a  digital  signal  indi¬ 
cating  which  crewmenber  is  talking. 

b.  Synchronization. 

Audio  data  must  be  synchronized  with  other  data 
recording  to  permit  relating  auditory  information  to  corre¬ 
sponding  performance  changes.  Each  recording  device  must 
therefore  include  an  audio  voice  track. 

c.  Data  Seduction. 

Manual  functions  in  audio  data  processing  are 
unavoidable,  but  can  be  minimized  through  computer- 
assistance.  Audio  data  playback  must  be  accompanied  by 
display  of  performance  parameters,  especially  the  parameter 
of  TIME,  to  allow  computer  correlation  of  manual  audio-data 
entry  with  the  digital  data  base.  A  convenient  means  for 
manual  data  entry  is  a  desirable  feature. 

These  data  processing  problems  must  be 
considered  in  the  design  of  a  combat-crew  training  measure¬ 
ment  system.  Thus,  since  HAF  has  at  most  the  same  aircraft 
as  the  aircraft  of  the  USA?  bases  from  which  data  were 
gathered,  HAF  must  apply  the  same  concepts  for  the  purpose 
of  deriving  combat  crew  performance  measurement.  This 
approach  will  solve  its  problem  of  continuously  optimizing 
the  combat  readiness  cf  its  combat  pilots.  These  concepts 
(measurements)  that  were  discussed  in  this  chapter  are  the 
following: 


(1)  Measurement  Commonality 

(2)  Common  Measurement 

(3)  Prototype  Measurement 

{4}  Measurement  Parameters 

(5)  Measurement  Descriptions 

(6)  Communication  Analysis 

Of  course,  the  value  of  experience  in  ccmbat  is 
not  easily  determined.  History  says,  experienced  fighter 
pilots  have  done  better  than  inexperienced  pilots  in  combat 
situations.  Intuitive  judgement  says  that  the  greater  the 
experience  level,  the  greater  the  chance  a  unit  has  for 
success. 


Modern  fighters  are  only  as  effective  as  the 
pilots  who  fly  them.  The  HAF  should  continuously  evaluate 
the  need  for  experienced  fighter  pilots  in  the  combat  ready 
units.  The  price  may  he  high,  but  the  price  of  defeat  in  the 
next  battle  for  the  air  may  be  even  greater.  At  the  end  of 
WWI  (World  War  I)  a  doctrine  was  written,  "...  if  you  hold 
the  air  you  cannot  be  beaten,  if  you  lose  the  air  you  cannot 
win."  [Bef.  28]. 


So  far,  in  order  to  establish  criteria  for 
combat  readiness  of  HAF's  combat  pilots,  we  have  dealt  in 
chapter  II  with  the  principles  of  human  performance,  in 
chapter  III  with  the  criteria  generally  and  their  measure¬ 
ment,  in  chapter  IV  with  the  introduction  of  the  combat- 
ready  crew  performance  measurement,  and  finally  in  chapter  V 
we  derived  the  combat-crew  performance  measurement 


ormance 


VI.  COSCIOSIONS  AND  recommendations 


The  Hellenic  Air  Force  has  always  been  interested  in 
keeping  the  highest  level  of  performance  for  its  combat 
pilots  and  keep  up  with  the  latest  of  technology  achieve¬ 
ments.  Studies,  as  that  of  OSAF,  have  always  been  of  a  great 
concern,  for  the  purpose  that  new  technigues  and  theories 
can  be  revealed  in  its  benefit.  In  this  way,  HAF  keeps  up 
with  the  latest  and  increases  the  efficiency  and  effective¬ 
ness  of  its  combat  pilots.  The  necessity  of  improving 
training  performance  information  of  its  pilots  reguires  new 
methods  of  measuring  flight  crew  performance  during  combat 
crew  training.  As  we  have  seen  in  this  study,  criteria  for 
combat  readiness  of  fighter  pilots  are  : 

(1)  Parametric  referent  or  standard  of  performance. 

(2)  Parametric  limit  about  the  parametric  standard. 

(3)  System  component  criteria. 

(4)  Test  criteria. 

(5)  Multidimensional  in  nature  and  represent  the 
complete  desired  end  result  of  a  system. 

From  the  examination  of  this  and  the  USAF  study  we  can 
conlude  that: 

(1)  Opening  a  new  flow  of  information  can  have  a  major 
influence  on  training  technology.  The  execution  of  effective 
contemporary  training,  the  development  of  new  training 
devices,  and  the  exploitation  of  powerful  concepts  such  as 
adaptive  training,  and  learner-centered  instruction,  all 
depend  on  information  available  through  performance 
measurement. 

(2)  The  role  of  performance  measurement  may  be  that  of 
causal  input,  a  catalyst,  or  a  weak  link  in  a  system  chain, 
tut  the  net  effect  of  better  performance  measurement  in  any 
case  is  a  positive  and  possible  revolutionary  improvement. 


(3)  The  major  difficulty  that  stands  in  the  way  of 
improvements  through  performance  measurement  is  that 
adequate  performance  measurement  systems  do  not  exist  where 
needed.  The  present  study  has  defined  a  guide  for  measure¬ 
ment  in  a  manner  permitting  adaptation  to  specific  needs  and 
budgets. 

(«)  The  status  of  combat-crew  measurement  and  ether 
areas  cf  complex  man-machine  performance  is  reflected  by  the 
measurement  analysis.  The  measurement  described  here 
reflects  the  critical  dimensions  of  performance,  and  agrees 
with  the  measurement  structure  used  by  operational  training 
personnel.  Given  this  capability  a  number  of  future 
advances  are  possible.  First,  use  of  this  measurement  will 
lead  to  efficient  interim  and  immediately  available  measure¬ 
ment  for  training.  Second,  improvements  in  the  generation 
of  optimally  efficient  measurement  sets  may  be  expected. 
Third,  clarification  of  the  relationship  between  objective 
and  subjective  measurement  should  be  possible.  None  of 
these  results  is  going  to  occur  without  an  acceptable 
performance  measurement  system  and  appropriate 
experimentation. 

This  section  is  concluded  with  the  following  recommenda¬ 
tions  for  future  development  opportunities  of  the  HAF: 

(1)  To  consider  measurement  in  the  context  of  combat- 
crew  training. 

(2)  To  assess  measurement,  that  is  to  identify  potential 
measurement  indicated  by  combat-crew  training  personnel. 

(3)  To  assess  the  constraints  placed  by  the  environment 
on  feasible,  usable  measurement  systems. 

(tf)  To  establish  the  model  of  instructional  system 
development. 

(5)  To  establish  a  measurement  system  for  the  opera¬ 
tional  environment. 


To  establish  an  automated  measurement  system  that  would 
relieve  the  instructor  pilot,  to  a  maximum  extend,  from  the 
requirements  of  having  to  record  a  great  deal  of  information 
manually  on  the  basis  that  such  activity  degrades  his 
ability  to  competently  instruct  his  student. 

(6)  For  the  purpose  of  deriving  combat-crew 
performance  measurement  to  consider  (a)  measurement 
commonality,  (b)  prototype  measurement,  (c)  measurement 
parameters,  (d)  measurement  specifications,  (e)  communica¬ 
tion  analysis. 

(7)  If  HAF  has  tc  make  a  choice  between  video/photc  and 
digital  recording  approaches  to  measurement  system  design, 
then  the  video/photc  recording  could  be  chosen,  as  a 
performance  measurement  system,  because  of  its  low  cost, 
information  provided,  flexibility  and  ease  of  use.  A  hybrid 
system  combining  the  advantages  of  both  is  preferable  to 
either  type  of  recording  alone.  Due  primarily  to  cost,  the 
bulk  of  measurement  parameters  would  be  derived  from  a 
video/photo  system,  and  the  remainder  with  a  small  digital 
recording  capability. 


APPENDIX  A 

P RCTOTYPE  MEASOBEMENT 

In  this  Appendix  an  example  of  prototype  measurement  is 
presented  for  takeoff  and  climb.  The  format  used  is  to 
present  a  discussion  together  with  prototype  measurement, 
indicating  through  the  Figure  A.1  the  types  of  information 
which  are  considered  important  to  a  description  of  pilot 
performance. 

TAKEOFF  S  CLIMB 

All  aircraft  takeoff  and  climb  to  a  cruising  altitude 
and  configuration.  Fixed-wing  aircraft  perform  these  maneu¬ 
vers  in  basically  the  same  way;  however,  at  a  detailed  level 
there  are  distinct  differences  between  aircraft.  Thus, 
measurement  must  be  tailored  to  each  aircraft,  but  the 
general  structure  of  such  measurement  may  be  defined  so  that 
the  essential  elements  are  constant  across  aircraft.  The 
following  sequence  is  rather  basic:  Takeoff  roll.  Rotation, 
Liftoff,  Gear-up,  Flaps-up,  Climb  and  Level-off.  The  infor¬ 
mation  desired  within  each  of  these  flight  maneuvers  may 
also  be  expressed  in  a  substantially  common  manner. 

Conditions.  To  properly  interpret  measurements  made 
during  a  particular  flight,  information  on  the  conditions 
existing  at  the  time  are  needed.  The  gross  weight,  wind 
direction  and  velocity,  runway  direction  and  length, 
temperature,  altimeter  setting,  field  elevation,  and  posi¬ 
tion  of  the  aircraft  in  formation,  are  reference  data  for 
the  evaluation  of  performance. 

Takeoff  roll.  The  takeoff  will  be  assumed  to  begin  with 
the  application  of  power.  The  takeoff  roll  maneuver  will  be 
considered  finished  at  rotation.  The  objective  is  to 
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TAKEOFF  5  CLIMB* 


CONDITIONS: 

"Cross  !7t! _  Bing:  _/  _  Runway:  _ / 

Temp.: _  "Alt.Set.:  "Field  Elev.  Form  Pos. : 


TAKEOFF  ROLL:  (To  rower  until  rotation) 

“"Power  Set!“  CentefTme“Pev.  :  Min,  Max,  Av. 

Reject  Speed!  Computed  Heading:Min,  Max,  Av. 

Time: _  Dist:  _  Bank:  5  Max,  L  Max 


ROTATICN:  (Nose  qear  off  until  pitch  att.  established) 

“Pof7~SpeedT - - StaE.  Trim: “I - - - 

Pitch:  Rate:  _ _ _  Bank:  _  ~ 

Final! “ _ CenterXine“ITev.  : 

Overshoot:-  Heading:  _ 2 


LIFTOFF:  (Pos.  Vert.  Vel.) 

“Xirspeed:  Pitch:  Bank:  Hdg: 

Vert.  Vel!  Ifter: _  Sec.:  ““ 


GEAR-OP: 

“Cear-Pp 

Pitch: 


FIAES  OP:  (Start  up  to  full  up)  ** 

Trim!  — - - "B^SZ^Only  IAS  PITCH  ALT  VV 

Pitch: _ Bank: _  Hdg: _  Start  x  x  x 

1st  Pos  XXX 
2nd  Pos  xxx 
Full  xxx 


*  11  •  uuu  m UUM 

A/S  (ITTXT)  771 NAL 

VV  (INIT1  —  (FINAL 

ALT  (INIT  J~  (FINAL 


x 

X 

X 

X 


TRIM 

X 

X 

X 

X 


CLIMB  S  LEVEL  OFT:  (Depends  on  Flight  Plan) 


Accelerate 
Climb  A/S  (#1) 

1*2}  x 

Climb  MACH  x 

Level-off  (Alt-10*  VV) 
(to  Cruise)  x 


INIT  FINAL 

PBR  A/S  MACH  HDG  ALT  ALT  PITCH  TRIM 


X 

X 


X 

X 


X 

X 


X 

X 


X 

X 


X 

X 


X 

X 


*  Also,  mandatory  communication  &  instances  where  A/C 
limits  are  exceeded. 

**F-106  has  no  flaps 


Figure  A. 1  Example  of  Prototype  Measurement 


accelerate  in  a  straight  line  along  the  centerline. 


or 


parallel  to  the  centerline,  with  wings  level.  Power  and 
resultant  acceleration  must  be  checked;  for  heavy  aircraft 
and/or  short  field  takeoffs,  accelaration  checks  are 
formally  performed.  lime  and  distance  along  the  runway  are 
checked  against  airspeed  to  determine  if  necessary  accelera¬ 
tion  performance  is  lacking  in  the  time  to  safely  stop  the 
aircraft.  Reject  speed  is  noted  in  case  of  an  emergence.  The 
formation  flight  leader  must  slightly  reduce  power  to  allow 
a  margin  of  thrust  control  for  other  members  of  the  flight. 

Rotation.  Proper  rotation  is  normally  necessary  to 
achieve  predicted  takeoff  performance.  Rotation  will  be 
defined  as  the  activities  between  the  time  that  the  nose 
gear  lifts  off  the  runway  until  the  time  that  a  stable  pitch 
attitude  is  established.  Stabilizer  trim  is  important,  bank 
angle,  centerline  and  heading  deviations  should  be  small. 
Rotation  should  occur  within  1-2  KIAS  of  the  desired  rota¬ 
tion  speed.  The  rate  of  rotation  should  not  be  either  too 
large  or  too  small.  A  specific  pitch  attitude  should  be 
established  witout  overshoot  or  oscillation. 

liftoff.  Liftoff  is  a  dicrete  event,  occurring  when 
vertical  velocity  is  positive.  At  this  time,  the  airspeed, 
pitch  angle,  bank  angle,  and  heading  are  noteworthy.  The 
vertical  velocity  a  short  time  after  liftoff  may  also  be 
measured  to  indicate  whether  the  aircraft  is  positively 
airborne,  or  if  there  is  any  tendency  to  settle  back  to  the 
runway. 

Gear-up.  Measurement  should  be  taken  from  the  time  that 
the  gear  handle  is  raised  until  the  time  that  the  landing 
gear  are  up  and  locked.  The  initial  speed  at  which  the  gear 
are  raised,  the  change  in  vertical  velocity  during  the  time 
that  the  gear  are  coning  up,  and  pitch,  bank,  and  heading, 
should  be  measured. 


Flaps-up.  Flaps-up  measurement  is  treated  in  somewhat 
the  same  manner  as  for  gear-up,  for  the  tasks  are  somewhat 
the  same:  a  configuration  change  is  occurring  which 
presents  a  perturbation  in  longitudinal  control.  A  trim 
change  occurs,  and  pitch,  bank,  and  heading  must  be 
controlled.  Normally,  flaps  must  not  be  raised  before  a 
specific  altitude  and  airspeed  (but  before  maximum  flaps 
speed),  and  during  the  transition  to  flaps-up,  changes  in 
airspeed,  vertical  velocity,  and  altitude  indicate  whether 
the  maneuver  is  properly  performed. 

The  B-52  presents  a  special  measurement  requirement 
since  a  specific  speed  schedule  must  be  maintained  as  flaps 
are  raised;  in  addition  to  airspeed,  pitch  angle,  altitude, 
vertical  velocity,  and  stabilizer  trim  are  of  interest 
during  this  period  of  time. 

Climb  and  Level-off.  For  each  aircraft,  there  are  a 
number  of  methods  for  climb-out  depending  on  the  flight 
plan,  and  desires  for  economy  or  performance.  It  may  be 
desirable  to  measure  climb  performance  from  liftoff,  cr  to 
start  when  the  aircraft  is  in  a  clean  configuration.  This 
phase  may  be  divided  into  the  following  parts:  acceleration, 
maintain  climb  airspeed  (may  be  several  increases  in 
airspeed  during  the  climb) ,  maintain  climb  Mach  number,  and 
level-off  (normally  level-off  begins  at  an  altitude  which  is 
below  cruise  altitude  by  10%  of  the  vertical  velocity) . 
Power,  airspeed,  Mach,  heading,  initial  and  final  altitude, 
pitch  angle,  and  trim,  are  parameters  which  may  be  measured 
during  each  portion  of  climbout. 
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